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Abstract 

Evolution depends on the possibility of successfully explor- 
ing fitness landscapes via mutation and recombination. With 
these search procedures, exploration is difficult in "rugged" 
fitness landscapes, where small mutations can drastically 
change functionalities in an organism. Random Boolean net- 
works (RBNs), being general models, can be used to ex- 
plore theories of how evolution can take place in rugged land- 
scapes; or even change the landscapes. 

In this paper, we study the effect that redundant nodes have 
on the robustness of RBNs. Using computer simulations, we 
have found that the addition of redundant nodes to RBNs in- 
creases their robustness. We conjecture that redundancy is 
a way of "smoothening" fitness landscapes. Therefore, re- 
dundancy can facilitate evolutionary searches. However, too 
much redundancy could reduce the rate of adaptation of an 
evolutionary process. Our results also provi de supporting ev- 
idence in favour of Kauffman's conjecture I Kauffman, 2000 
p.195). 

Introduction 

A system is robust if it continues to function in the face of 
perturbations (Wagner, 2005a I. This makes robustness a de- 
sireable property in any system surrounded by a complex 
environment. Living systems fall into this category. Sys- 
tems able to cope with perturbations will survive and repro- 
duce. Moreover, robustness plays a key role in evolvabil- 
ity: it allows the gradual exploration of new solutions while 
maintaining functionality. A small change in a fragile sys- 
tem would destroy it. This is not favoured if the exploration 
method consists of random mutations. 

There are several mechanisms that help a system 
to be robust ( |von Neumann, 1956) , such as modular- 
ity HSimon, 19961 IWatson and Pollack, 2005) , degen- 
eracy (Fernandez and Sole, 2004), distributed robustness 
(Wagner, 2004), and redundancy. Here, we focus on this last 
one. It consists on having more than one copy of an element 
type. If one fails or changes its function, another can still 
perform as expected. Redundancy is widespread in genomes 
of higher organisms ( |Nowak et al., 1997| ). A clear example 
is seen with diploidy, where each cell has two complete sets 
of genes. This has several advantages for evolvability, e.g. 



it provides robustness to recessive detrimental mutations of 
one gene of a pair. There is also substantial evidence for 
the hypothesis that growth in genetic regulatory networks 
occurs primarily via the duplication of genes and subsequent 
mutations of one or both members of the duplicate pair 
( Fost er et al., 2 005 ; Lewin, 2000). Thus, it would be desire- 
able to obtain a better understanding of its mechanisms. To 
achieve this, we used random Boolean networks (RBNs) 
(IKauffman, 19691 IKauffman, 19931 |Wuensche,"?998l 
lAldana-Gonzalez et al., 20031 IGershenson, 2004) , a very 
popular model of genetic regulatory networks. 

Our present work was initially motivated by Kauffman's 
conjecture ( Kauff man, 2000| p. 195). This states that 1- 
bit mutations to a minimal program will change drastically 
the output of the program, making it indistinguishable from 
noise. The conjecture points to the necessity of having 
some redundancy to allow smooth transitions as a program 
changes in an evolutionary search. 

However, it is not obvious to obtain a minimal pro- 
gram in a standard programming language such as C 
or Java. Still, we can consider cellular automata (CA) 
( |von Neumann, 1966) or RBNs as programs. It has been 
shown that certain CA can perform universal computations 
( |Gardner, 1983l|Cook, 2004) . Since RBNs are a well stud- 
ied generalization of CA ( |Gershenson, 2002) , they seem 
suitable candidates for studying the effect of redundancy 
in their robustness. We can measure the redundancy of a 
RBN, make 1-bit mutations to the networks, and then mea- 
sure their robustness comparing the state spaces of the mu- 
tant and the "original" networks. 

It is not obvious how to measure redundancy, or 
compressibility. Interpreting results from Kolmogorov, 
Solomonoff, and Chaitin l |Chaitin, 1975| ) we can say that it 
is impossible to show that a program is "minimal" (inde- 
pendently of a fixed universal model of computation). We 
decided not to attempt to build "minimal" networks, but to 
compare more and less redundant networks, and try to find 
differences. 

This paper is organized as follows: In the following sec- 
tion, we present random Boolean networks. Then we intro- 



duce a method for adding redundant nodes in RBNs. We 
used this method in simulations to measure the robustness 
of RBNs to point mutations. A discussion of our results fol- 
lows, to finally present concluding remarks. 



Random Boolean networks 

A random Boolean network (RBN) 



(Kauffman, 1969 




Kauffman, 1993 




Wuensche, 1998 
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N nodes (consisting each of one Boolean variable) 
{g\,G2, ■ ■ ■ ,<5n} that can take values of zero or one. The 
state of each node o, is determined by k[ randomly chosen 
nodes (connections) {o, p - - ,CJ, t .}. In the classic model, 
kj = kj, V so every node has K inputs. 

The way in which the kj elements determine the value 
of o, is given by a Boolean function //((J/j , ■ ■ • ,Gi k .)- Each 
combination of inputs (there are 2 k ') will return fi = l with 
a probability p and /, = with a probability 1 — p. Once the 
connections and functions have been chosen randomly, the 
network structure does not change. In the classical model, 
the elements are updated synchronously: 

m(t+i)=f t (sj h (;),•• •,%(*)) (i) 

fi can be represented as a lookup table, where the right- 
most column represents a, (t + 1) determined by ki inputs 
represented in the rest of the columns exhaustively com- 
bined in 2 k ' rows. 

Since the number of possible states is finite (2 N ), sooner 
or later the network will reach a state that has been already 
visited. Because the dynamics are deterministic, the net- 
work has reached an attractor. If we use RBNs as models 
of genetic regulatory networks, attractors can be seen as cell 
types (Huang and Ingbe r, 2000) . 

The dynamics of "families" or ensembles 
( Kauffma n, 2004) of networks can be studied to find 
statistical properties common in networks independently 
of their precise functionalities. We will use this ensemble 
approach to study the effect of redundancy in families of 
RBNs. 

Introducing Redundancy 

At first we attempted to study the effect of redundant links in 
RBNs. However, redundant links are fictitious links i.e. the 
functionality does not change when they are removed. We 
began with a network with a desired percentage of fictitious 
inputs (this can be regulated with probability p of having 
l's in lookup tables), and then measured its stability. After- 
wards we reduced the number of fictitious inputs, measuring 
the stability until there were no redundant links. The way 
we measured stability was: first, sensitivity to initial con- 
ditions, comparing if similar initial states converged or di- 
verged. Second, making 1-bit mutants of the lookup tables 
of the networks, and then comparing the overlap of the state 



space transitions using normalised Hamming distances (0. 
Our result: there was no difference. First, since fictitious 
inputs have no functionality, they do not affect the conver- 
gence of the networks. Second, it seems that the probability 
of making a mutation to a redundant link (which did not have 
functionality) and making it functional is the same as do- 
ing the opposite (making a functional link non-functional). 
Therefore, the redundancy of links does not affect the stabil- 
ity of networks. 

1 " 

H(A,B) = -J2\a i -b i \ (2) 
n *■ — ' 

i 

We realized that the redundancy in nodes was very differ- 
ent from the redundancy in links. 1 It would be computation- 
ally complicated to generate a network with some redundant 
nodes, and then trim them. Therefore we did the opposite: 
RBNs were generated randomly, and then redundant nodes 
were added. 

The method we developed for adding a redundant node to 
a RBN is the following: 

1. Select randomly a node X to be "duplicated". 

2. Add a new node R to the network (N new = N" ul + 1 ), with 
the same inputs and lookup table as X (i.e. k]( — kx,fR — 
fx), and outputs to the same nodes of which X is input: 

k ne W=k old uk . R if3k . x yi (3) 

3. Double the lookup tables of the nodes of which X is input 
with the following criterion: When R — 0, copy the old 
lookup table. When R = 1, and X = 0, copy the same 
values for all combinations when X = 1 and R = 0. Copy 
again the same values to the combinations where X = 1 
and R = 1 . In other words, make an inclusive OR function 
in which X OR R should be one, to obtain the old outputs 
when only X was one 

f'"' (o (l , ■ ■ ■ , <% = 0, s is = • • • , a, ) = ft 14 (a/j , ■ ■ ■ , a tx = 0, • • • , Ofj . ) 
/T'(«'ii.-,Oix=0,o ii! =l...,o^)=^«(o il ,-,o, x = l,...,a ilj ) ( 
f;«"-(e ir ....a ix = l.a ill =0--,a iti )=f? d (c, 1 ,-...c ix =L--,a ik .) ( '** 

fT (°<i > ' ■ ■ ' <% = 1 ' a 's = 1 ' ' ' ' % ) = if* (%.■■■, <% = h — . Tj, ) 

After this, R will be a redundant node of X, and vice versa. 
We will call R, and other redundant nodes which are added 
to the original network, "red" nodes. The "original" nodes 
of the network will be called "white" nodes. A diagram il- 
lustrating the inclusion of a red node is depicted in Figure^ 
Lookup tables show how the node of the rightmost column 
(in bold) at time t + 1 will be affected by the values of the 
other columns at time t . 

We implemented this algorithm in RBNLab 
( Gershenson, 2005 ), and used this software laboratory 
to measure the overlaps of state space transitions of 1-bit 

'This was in spired in redundancy of evolvable hardware 
(Thomp sonTT998) 



Figure 1: Example of addition of a red node to a RBN N = 3, K = 2 (see text for details) 



mutant nets with "original" ones, as we add more and 
more redundant nodes. The mutations consist in flipping a 
random bit from the lookup table of a randomly selected 
node. We used normalised Hamming distances (|2j to 
measure the difference dS between state spaces: 

2" 

dS=^ i Y j H{A> j +\M>+ i ) (4) 

i 

where A is the "original" network, and M is a 1-bit mu- 
tant of A. dS is calculated by computing one time step in 
both networks for each initial state i. Then, the difference of 
the states that both networks transitioned is calculated with 
(0. This is averaged for all initial states (2 ), and the result 
reflects how similar the transitions are in both state spaces. If 
dS = 0, there is no difference between state spaces, and thus 
the mutation had no effect. A higher dS reflects a greater 
effect of the mutation in the state space. There is no cor- 
relation of state spaces, i.e. a mutation is maximally catas- 
trophic, when dS ~ 0.5. 

A disadvantage of this method is that it is restricted to 
small networks, since the state space is 2 N . Therefore, for 
each node that is added to a network, its state space is 
doubled. Another option would be to have non-exhaustive 
explorations of state spaces, but this could be misleading. 
Therefore, we decided to limit our simulations to small net- 
works. 

Simulation Results 

We can appreciate the average results of ten thousand net- 
works of N = 7, p = 0.5 and different K values in Figure 
|2] We can see that dS decreases exponentially as red nodes 
are added. We should note that the dS for RBNs without red 
nodes decreases as K increases. This is because the lookup 



tables are doubled each time K is incremented. Thus, a one- 
bit mutation will have less effect on a network with higher 
connectivity. To overcome this, we plotted the values of dS 
for the same results in Figure [5] We can see that a 1-bit 
mutation makes dS ~ 0.1 for networks without red nodes. 
Clearly the addition of red nodes increases the robustness 
of the networks. The effect of red nodes is more evident 
for higher K values, where the network dynamics are more 
chaotic (Gershenson, 2004 1. 

To be certain that the robustness is given by the addition of 
a red node, and not of any node, we performed simulations 
with N = 10. The average values of dS for one thousand 
networks are shown in Figure @] We can quickly see that 
a network with seven white nodes and three red nodes has 
a lower difference in state spaces than a network with ten 
white nodes and no red nodes, especially for high values of 
K. 

We obtained results of RBNs with scale free topology 
( Aldana, 2003 1 (data not shown). We observed a similar ten- 
dency: as red nodes are introduced in a network, mutants 
will have less differences in the state space, especially for 
chaotic networks of high connectivity. 

We also obtained results for different updating schemes 
( Harvey and Bossomaie r, 1997| IGershenson, 2002 

iGereheTisoTetaTH 2003) and" both topologies, with similar 
results (data not shown). The precise effect of redundant 
nodes on dS does change with the updating scheme: in all 
cases dS decreases as redundant nodes are added. 

Discussion 

It is intuitive to assume that redundancy will increase the ro- 
bustness of a system. If in a minimal program all nodes mat- 
ter, any mutation will have more drastic effects in the output 



Figure 4: N = 10, average dS for 1000 networks 



than in a non-optimal program. This has been seen exper- 
imentally in evolvable hardware (Thompson, 1998), where 
some redundancy allows better evolution of circuits, but too 
much redundany obstructs it. Still, the goal of our work was 
to quantify the effect of node redundancy in robustness to 
mutations in RBN. 

Fictitious inputs do not increase the robustness because 
mutating such inputs can change the logic function. This 
is not the case for redundant nodes: if a node is mutated, 
the rest of the network keeps its (partial) function. In 
other words, mutations in redundant nodes do not propagate 
through the network, whereas mutations in fictitious inputs 
do. 

From our results we can say that redundancy smoothens 
rough fitness landscapes (Kauffman, 1993), making evolu- 
tionary search potentially feasible in rough fitness land- 
scapes. However, too much redundancy should be detrimen- 
tal for evolvability, because evolutionary search becomes 
slower. We can also confirm recent results by Andreas Wag- 
ner ( |Wagner, 2005b) and notice that red nodes increase the 
neutrality (Kimura, 1983) of the network. A neutral muta- 
tion is one that has no effect in the function (or phenotype). 
As we can see, redundancy of nodes decreases the probabil- 
ity of a mutation causing a change in the functionality. If a 
mutation occurs in a red node, another node will take its role 
to perform the old function. From this we can suggest people 
working with genetic algorithms: if a fitness landscape is too 
rough to make an evolutionary search on it, then add some 
redundancy that will smoothen it. Nevertheless, not all types 
of redundancy will be useful ( |Harvey and Thompson, 1997) . 

It could be argued that "useless" redundancy can be added 



to a state space, i.e. with no effect on the function, and thus 
would be of no use for evolvability. However, in our RBN 
approach, a red node cannot be useless, since it is connected 
to the rest of the network and thus affects its behaviour. 
Therefore, any red node will have a potential effect on the 
function of a mutated network: the function of the mutated 
network will be close to that of the original network, for an- 
other node is duplicating the functionality that changed with 
the mutation. In other words, red nodes provide "useful" re- 
dundancy. On the other hand, redundancy given by fictitious 
links in lookup tables is indeed "useless": adding fictitious 
links will have no effect on the mutated network function- 
ality. Still, a general mechanism for deciding wether a cer- 
tain type of redundancy is useful or not seems to be an open 
question. In our case, node redundancy does help an evolu- 
tionary search in rugged fitness landscapes. To which cases 
this is applicable remains to be explored. For example, if a 
fitness landscape is already smooth, adding redundant nodes 
will be useless to an evolutionary search. 

Returning to the original motivation of this work, if 
Kauffman's conjecture holds, a random mutation on a "min- 
imal" RBN should produce adS ~ 0.5. However, we cannot 
prove if a RBN cannot be reduced or compressed indepen- 
dently of a fixed universal model of computation. Thus, we 
cannot know if a RBN is minimal. What we have seen is 
that "less compressible" RBNs, i.e. with no red nodes, have 
always a higher dS. As we add more red nodes, the RBN 
becomes more compressible, and dS decreases, showing an 
increase in robustness. As we have seen, the compressibil- 
ity is directly proportional to the robustness of a network to 
random mutations. We could extrapolate and induce that if 



the compressibility could be minimal, then the robustness 
would be also minimal. The minimal robustness would be 
given when a single mutation would create dS ~ 0.5, so we 
can conclude that this corresponds to a minimal compress- 
ibility, i.e. a minimal program, even when in theory it is 
not possible to show this. This does not prove Kauffman's 
conjecture, but provides reasonable evidence for its validity. 

Conclusions 

We have presented an algorithm for introducing redundant 
nodes in a random Boolean networks. We used this to show 
that redundancy of nodes increases the mutational robust- 
ness of RBNs. This may have advantages for evolvabil- 
ity, depending on the particular problem domain. How- 
ever, there are other issues that should be also considered 
for evolvability, such as distributed robustness, degeneracy, 
and modularity. The relationship between neutrality and re- 
dundancy demands further study. This is also the case for 
redundancy of nodes in larger networks. 
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